86

7

The Transmission of Information

particularly common in condensed matter: any glass, for example, breaks ergodicity.

In nonergodic systems, the phase space or ensemble average does not equal the time

average.

A homely illustration of some of the issues to be considered, in particular that

breaking ergodicity depends on the timescale of the observer, is provided by a cup

of hot coffee to which cream is added and stirred. The coffee and cream become

homogeneously mixed within seconds, the cup and contents reach the temperature of

the surroundings after tens of minutes, and the water evaporates and is in equilibrium

with the atmosphere in the room after many hours. Whether the observed behaviour

is representative of the allowed phase space depends on the observational timescale

tau 0τ0. In general, broken ergodicity can be expected if there are significant dynamical

timescales longer than tau 0τ0.

In a more general sense, applicable also to symbolic strings, ergodic means that any

one exemplar (substring) is typical of the ensemble; hence, if the string is ergodic,

it is to be expected that every permissible sequence will be encountered. Clearly,

therefore, the DNA of living organisms is not ergodic (although it might be argued

that hitherto we have taken a too liberal view of what is “permissible”).

7.5

Noise

So far we have supposed that the messages received over the communication chan-

nel are precisely those transmitted. This is a rather idealized situation. We have

doubtlessly had the experience of speaking on a very noisy telephone line, or listen-

ing to a radio with very poor reception, and only been able to make out one word

in two perhaps, and yet could still understand what was being said. The syntactical

redundancy of English is about 0.5; hence, it is not surprising that about half the

words or symbols may be removed (at random) without overly impairing our ability

to receive the original message.

According to our previous discussion of the Shannon index, upper II is additive for

independent sources of uncertainty. Noise is an independent source of uncertainty

and can be treated within the theoretical framework we have discussed.

Suppose that signal xx was sent and yy was received, the difference between the

two being due to noise. The amount of information lost in transmission is called the

equivocation, upper EE.

Definition. The equivocation is

upper E equals upper I left parenthesis x right parenthesis minus upper I left parenthesis y right parenthesis plus upper I Subscript x Baseline left parenthesis y right parenthesis commaE = I (x) −I (y) + Ix(y) ,

(7.11)

whereupper I left parenthesis x right parenthesisI (x) is the information sent,upper I left parenthesis y right parenthesisI (y) is the information received, andupper I Subscript x Baseline left parenthesis y right parenthesisIx(y) is the

uncertainty in what was received if the signal sent be known. ¹¹

11 It should be clear that the information sent is already the result of some measurement operation

or whatever, in the sense of our previous discussion.